High-speed digital communication networks over copper, optical fiber and other hybrid media are used in many digital communication systems and storage applications. As these networks continue to evolve in order to meet ever-increasing bandwidth requirements, new protocols are being developed to more efficiently transfer information throughout these networks. For example, the well-known IEEE P802.3ae Draft 5 specifications describe requirements for 10 Gigabit Ethernet (GbE) applications, which may be used in communication networks and also in storage area networks (SANs).
Notwithstanding, the processing power and memory bandwidth of networking devices used in applications such 10 GbE have generally lagged behind the increased demand for networking bandwidth and faster data rates. In networks such as the Internet, which utilize transmission control protocol/internet protocol (TCP/IP), data copying and data transferring operations utilize a great deal of CPU and memory resources. In addition to diminished processing capacity, copying and transferring operations may also increase latency and delays in data processing, which may significantly effect time critical applications such as voice applications. A major consequence is that network computing nodes have become bottlenecks, which may significantly diminish system performance and network throughput.
Some TCP/IP networks typically employ a TCP offload engine (TOE) to facilitate more efficient packet processing. A TOE is an intelligent network adapter or network interface card (NIC) that may be configured to offload most or all of the TCP/IP protocol processing from the processor or CPU of a host system to the network adapter. One of the primary challenges associated with building a TCP offload engine involves the handling of non-posting TCP applications running on a host. Non-posting TCP applications are those that do not post or allocate buffers before the network adapter will receive data for the application. Current TOE designs store all their received data locally on the adapter in buffers known as TCP segment buffers. When a TCP application buffer is finally posted on the host system, the network adapter copies the data from the TCP segment buffers to the posted TCP application buffers and subsequently releases the TCP segment buffers for re-use by the TOE.
Conventional offload NICs may store, in TCP segment buffers, incoming data that is received in-order or out-of-order (OOO) from the manner in which it was transmitted at the other end of the network. Data may be received out-of-order for a variety of reasons, for example, when a packet is retransmitted after a request for retransmission occurs, which may occur after some of the later packets in the original transmission have been received properly. Packets may also be re-ordered due to routing changes within the network. However, storing the out-of-order data in TCP segment buffers can be quite hardware intensive. The quantity of out-of-order data, and therefore the size of TCP segment buffers needed, may scale with, for example, the bandwidth of the network connections, the delay on the connections, the number of connections and the type of connections. As network connections grow larger and faster, the quantity of out-of-order data that may require storage may be significantly larger than what the NIC may be capable to store. Large numbers of incoming out-of-order data may also prevent or otherwise hinder the immediate processing of TCP control data performed by the NIC because of the added processing involved in managing and storing large quantities of data. Large numbers of incoming out-of-order data, after sitting in the TOE storage, may be complete when missing data arrives. Now the TCP segment buffer bandwidth and host memory bandwidth must be doubled to allow the delayed data to be written to the host along with any new in-coming data. In some instances, a TOE implementer or designer may be faced with the undesirable option of dropping out-of-order data if there is insufficient storage space on the NIC or even dropping in-order data due to lack of host or re-assembly buffer bandwidth. When out-of-order data is dropped or otherwise discarded, the sending side may have to retransmit all the dropped data. Since dropped data is interpreted as congestion by the TCP sending station, this approach may result in about a fifty percent (50%) decrease in throughput or network bandwidth utilization in some instances, effectively eliminating any of the benefits of offloading TCP applications to a network adapter in the first place.
Some network adapters may use generic buffers on the non-posting host for handling TCP data that has been received in-order until posted buffers may become available. This approach may provide some relief to the TOE in terms of hardware requirements. In such instances, the TOE simply sends the in-order data to the generic buffers on the host system as received. However, this approach does not address the scaling concerns that may be faced by network adapters for handling the storage and management of incoming out-of-order data when network connections become larger and faster. The issue remains that received out-of-order data may be too large to store in the network adapter. In the case that network adapters may utilize also generic buffers in the non-posting host for received out-of-order data, the non-posting host may have to maintain open and keep track of what may be a large number of generic buffers containing out-of-order data until all the necessary data has arrived and/or until a posted buffer is made available. In this regard, the non-posting host may, in some instances, not be able to copy portions of data stored in the generic buffers even when a posted buffer is made available. This result may arise since the non-posting host tracks all the received data, in-order and out-of-order, by means of the relative information in the generic buffers. Closing any of the generic buffers, even when the contents may be transferred to a posted buffer, may make tracking of all the out-of-order data much more difficult to perform. When multiple connections exists and the number of generic buffers increases significantly, the non-posting host may be limited in its ability to handle out-of-order data. Moreover, the non-posting host may not make efficient use of the memory resources available and system performance and network throughput may be significantly affected.
Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of such systems with some aspects of the present invention as set forth in the remainder of the present application with reference to the drawings.